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ABSTRACT 

The study investigates the potential utility of a 
predictor instrusent in the training of manual control operators in 
aircraft simulators. Various predictor di&jlay design configurations 
vere presented to subjects during training trials on an aircraft 
approach to landing task. Subsequently, subjects vere tested on 
trials devoid of the predictor instrument to determine transfer 
effects. Each of the predictor display configurations was contrasted 
with a control or baseline display which did not include the 
predictor^ Results of the study indicated that transfer from a 
predictor to a nonpredictor display was dramatic; learning was 
greatly accelerated, and the difference between pr.edictor trained and 
nonpredictor trained subjects appeared permanent, in view of a rather 
stable asymptotes observed during the last 214 test trials. It was 
concluded that the predictor display accelerated learning through its 
ability to provide immediate feedback to operators regarding the 
unique response of the system to control actions* Such feedback 
enabled operators to learn the complex dynamics of. the vehicle much 
more rapidly than was possible without it. Implications for use of 
the predictor instrument in wide variety of operational and training 
applications are discussed* Appended materials include a 
bibliography, a report on exploratory research, computer simulations, 
and subject instructions. (Author/BP) 
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I. INTRODUCTION 



A. PURPOSE 



BEST COPY AVAILABLE 



Use of a predictor display has been sixown to virtually transform the 



difficulty of a variety of complex, manual control, pursuit tracking tasks 
to the level of those having relatively simple control requirements. 
Within 15 minutes practice, naive operators have been able to perform 
some complex tasks with a predictor display at accuracy levels pre- 
viously achievable only after extensive training without a predictor dis- 
play (Kelley, 1968). 

The purpose of the present study was to explore adaptive use of a 
predictor display to promote rapid and accurate learning on conventional 
tr^^king tasks, i.e., transfer of training. 

The study consisted of two phases. The remainder of this chapter 
presents a review and theoretical discussion of the problem (Section B), 
a summary of exploratory research conducted in Phase I (Section C), 
and a brief description of and underlying rationale for the experiment 
conducted in Phase II (Section O). Chapters II through IV describe the 
Phase II experiment in detail. 

B. BACKGROUND 
1. Introduction 



inaccuracies in system response will result. Such an effect results from 
man's limited capabilities to simulate the ideal servomechanism and to 
predict near future system control needs. An excellent example illustrating 
his limitations is a flight task. A large part of learning to fly involves 
learning to make appropriate control inputs in response to information in 
the visual world outside the cockpit and to information contained in the 



The more complex and difficult a manual control task, the more 
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flight instruments. The task is uniquely complex under IFR conditions, 
where system status information is, for the most part, restricted to 
flight instruments alone. 

In principle, to maintain appropriate aircraft control the pilot 
continously determines the present status of the aircraft, extrapolates 
its near future status .and then formulates and executes responnes that 
will hopefully maintain or bring the statu^,.of,J»e ,a^ within accer ta- 

ble tolerances. Inherent in this control process are several sources of 
time lags which contribute to the instability of the control system and 
difficulty of the control task. 

2. Time Lags 

One primary set of lags is inherent in the control dynamics of 
the aircraft. A step function input to the stick, for example, is integrated 
over time and appears several seconds later as an accelerating change 
in attitude (e. g. , Etkin, 1958). The aircraft is a high inertia system and 
cannot respond immediately to control commands. 

A second set of lags is associated with the perceptual and motor 
processes of the pilot. Changes in positions, velocities, accelerations 
and other parameters varying in time must exceed certain threshold 
values before they are perceived as having changed. If any of those rates 
of change are low, delays in detecting the changes will occur (Osgood, 
1953). Moreover, regardless of detection lag time, the pilot's minimum 
muscular reactior; time represents a significant system lag as well. 

Finally, a third set of lags is associated witli human information 
processing time. The amount of processing time required in any partic- 
ular situation depends on thd amount and complexity of information to be 
processed. Human information processing is relatively slow at best 
and when the operator's limited capacity is exceeded, delays occur and 
errors build up rapidly (e.g. , Knowles, Garvey & Newlin, 1953). 

•» 2 
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To maximize system response accuracy, it is necessary to 
compensate for lags characteristic of both control dynamics and human 
performance. Without such compensation, the system will be annoy- 
ingly oscillatory at best or fatally unstable at worst (Etkin, 1958). 

3. Lag Compensation 

In normal flight, the pilot provides some degree of compensa- 
tion, a capability he develops through training and experience in simu- 
lators and in ictual flight. However, while discussions oi i jdiction in 
manual control tasks in the literature often seem to suggest that opera- 
tors do employ predictions with some degree of success, the evidence 
supporting such suggestions seems nonexistent. In fact, what evidence 
there is suggests the opposite conclusion. For example, in a series of 
early studies employing a one-dimensional pursuit display; Gottsdanker 
(1952ab, 1955) observed no predictive capability whatsoever on the part 
of his Subjects. Moreover, Smith and Lyman (1967) found similar results 
using a high inertia, two dimensional tracking mount. Thus, it may well 
be that the human operator functions almost entirely as a slow servo- 
mechanism, as suggested long ago by Craik (1947, 1948). That is, the 
operator inputs a control action only after he perceives an error. Pre- 
cision manual control, therefore, is not possible in conventional systems 
because operator and system lags caimot be rapidly compensated for by 
the human operator. 

Compensation may be provided in other ways. Autopilot and 
stability augmentation control systems provide compensating lead terms 
automatically through servomechanisms tailored to the flight dynamics 
of the aircraft. Quickened displays, which apply the principles of stability 
augmentation to the display system, represent a technique used to provide 
compensation to the system by preconditioning the input to the pilot 
(Birmingham and Taylor, 1954). Predictor displays which utilize fast- 
time analogs of the control system dynamics, also provide compensation 
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by processing current aircraft status and integrating system dynamics 
equations to form predictions of the future state of the aircraft (Kelley, 
1962). 

Regardless of how compensation is pi^ovided, its purpose is to 
stabilize control of the aircraft so that it can be flown more easily, 
smoothly and safely through desired maneuvers. For systems which 
require, for one reason or another, manual control by the human opera- 
tor, predictor instruments appear to hold unique promise for acjhievring 
control precision. 

4. Predictor Displays , 

It has been amply demonstrated that predictor displays can 
enable operators to control systems in which the dynamics are so 
complex that they are virtually impossible to control with feedback 
from conventional displays (Kelley, 1962). It has also been demon- 
strated that the efficacy of predictor displays ts a function of several 
parameters such as prediction span, repetition rate, operator response 
model, or display format that can potentially serve as variables to 
be adapted during training to effect transfer from a predictor instru- 
ment to conventional instruments (Pitrella, Prosin, Kelley & Wulfeck, 
1971). 

As noted in Section 3, a predictor instrument utilizes a fast- 
time model of control dynamics to compute the future course of the 
system output, given the present state of the system and given certain 
assumptions about the operator's behavior. Since the real state of the 
system is sampled and the prediction calculations are repeated several 
times a second, the models for the control dynamics and operator per- 
formance can be crude but yet quite effective. For example, the 
dynamics of a pitch control system may be computed by sampling rela- 
tions between deflections of the stick and control surfaces, changes in 
angle of attack, and changes in attitude. The predictor model incorporate 
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the same elements scaled to operate many times faster and arranged so 
that the prediction is interrupted, reset, and restarted at a rate deter- 
mined by the desired prediction span and display refresh requirements. 

The output of a fast-t^me, repetitive model can be displayed in 
a number of ways. For present purposes, a simple display showing pre- 
sent position and predicted trajectory of an aircraft relative to a glideslope 
and a 0. 5° tolerance envelope above and below the glideslope is used (see 
Figure 1). 

The strategy is to control the aircraft such that present/position 
remains within the tolerance envelope and, as much as possible, super- 
imposed upon the glideslope. Since tho predictor trace is generated in 
fast-time, it responds to the control inputs several hundred times faster 
than the actual aircraft or aircraft symbol and gives the pilot virtually 
instantaneous feedback with respect to the "long term" effects of his con- 
trol actions. This feedback no<- only greatly simplifies the pilot's control 
task, it -also gives him information that would seem useful as instructional 
feedback (Roberts & Taylor, 1972). 

5. Predictor Instruments and Training 

Novice pilots nearly always over -control and end up in a highly 
oscillatory mode of operation. In the pitch dimension, stick position 
information is available from kinesthetic and visual cues, as well as 
from stick force feedback. In actual flying, position information is also 
available through proprioceptive (body angle) cues. Angle of attack is 
sometimes presented directly, but more often it is presented indirectly 
via pitch angle. Attitude is presented directly and also indirectly as 
vertical velocity. Of course, none of this information is sufficiently 
precise or rapid. Consequently, over -controlling occurs when the pilot 
leaves in a stick deflection until he perceives a resulting change in pitch 
angle or vertical velocity and then tries to restore the stick to neutral. 
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Figure 1. A cathode ray tube display showing an imprinted 
glideslope and tolerance envelope, and a simu- 
lated aircraft and its predicted trajectory. 



1 6** 



ERIC 



17 



However 9 the corrective action is usually too late and an opposite com- 
pensating control action must be introduced. 

The fast-time predictor display reduces over-controlling by 
providing immediate feedbacH-in terms of predicted outcome. And> as 
noted, it may also provide instructional cues to assist the student in 
learning to (1) understand the dynamics of his control system, (2) look 
for early derivative information, and (3) make small, intermittent 
step*' function control movements. Early in training, the student usually 
controls the end of the predictor trace; with practice he learns also to 
control the shape of the trace, which depends on proper control of the 
derivative terms* Again, the fast-time feature of the predictor model 
provides the instantaneous feedback needed to facilitate his learning. 

The evidence supporting the use of a predictor display for a 
wide variety of manual control systems is exceedingly powerful. How- 
ever, its widespread implementation appears likely to be slow, although 
one is already flying in the DC- 10. It awaits a thorough investigation in 
terms of the many parameters that constitute elements of its design, 
i. e* , the various parameters need to be varied and evaluated for specific, 
types of control tasks in order to establish design criteria for optimizing 
predictor displays for a wide range cf s/stems. In the meantime, however, 
the question has emerged as to whether a current predictor in8tr\mient can 
be used fruitfully as an aid to facilitate learning on conventional displays. 

6. Transfer of Training 

Among the many variations in training methods, it is possible to 

distinguish three major strategies. At one extreme is the whole-task 

approach in which the student is required to practice the actual task in 

its entirety until he masters it. Since the learner must ''sink or swim, 

this approach is best when (1) the task is already within the response 

capabilities of the learner when training begins, (2) the task is not 

particularly difficult, and (3) there are no overriding costs for errors 
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and failures. Thus, this mode of training is useful in improving the 
efficiency with which a learner may perform a task he can already do, 
but is quite inappropriate in training situations in which She task far 
exceeds the learner's initial capabilities and the purpose of training is 
to develop new skills and capabilities that enable him to handle the task. 

Two other general modes of training to develop new skills are 
those explicable in terms of the concepts of "transfer" and "shaping. " 
in transfer modes, the learner is set to a task the mastery of which 
presumably will facilitate the subsequent mastery of a new task. For 
example, listening to lectures, reading books, and practicing in simu- 
lators are all activities which must be performed at a specified level, 
witii the expectation being that tfiere will be positive transfer from these 
tasks to the ultimate task of flying an aircraft. A number of theoretical 
explanations exist which account for reasons why transfer takes place 
(see, for example, Clark, 1972), but, for all practical purposes, deter- 
mining whether the learning of one task will facilitate or inhibit sub- 
sequent learning of any given second task usually requires empirical 
investigation. 

The concept of "shaping" describes what is really a special case 
of the transfer mode of training, although it is usually not considered as 
such. Shaping is training by eliciting and reinforcing successive approxi- 
mations to the desired performance until the final desired performance is 
achieved. By making the steps between successive tasks very small, by 
programming the introduction of new task elements to capitalize on skills 
already developed, and by careful attention to the nature, contingencies 
and schedules of reinforcement, complex skills can be developed quite 
efficiently. Shaping is a special case of transfer since each trial is 
specifically designed to maximize the transfer from all the preceding 
trials (Gagne and Rhower, 1969). 
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C. EXPLORATORY RESEARCH BEST COPY AVAIL/IBLE 

A literature review, development of rationale for a transfer of ti'aiu- 
ing experiment using the predictor instrument, and exploratory research 
constituted elements of Phase I of the project. Section B presented finding 
from the literature search. The present section discusses rationale and 
exj»loratory research. 

1. Objectives and Rationale 

It was originally proposed that an experiment be conducted to 
investigate the effects of using the predictor display on learning to master 
an IFR final approach to landing task in which simulated instruments repre 
sented the conventional primary displays. The predictor display was to be 
used in an adaptive or "as needed" mode to allow operators opportunities 
to relate changes in instrument readings with predictions generated by the 
predictor display. The purpose was to determine whether availability of 
the predictor display would significantly reduce the time to learn to per- 
form a final approach task accurately using only the conventional 

» 

instruments. 

In brief, the reasons for believing that an adaptive predictor 
display would be useful in facilitating learning of such a complex flight 
control task were: 

• The predictor display greatly simplifies the effective 
dynamics of the pilot's control task and, hence, allows 
him to attend to otlier features of the overall task such 
as interpreting and integrating information from the 
conventional instruments. 

• The adaptive feature can be used to force a reduction 
in the pilot's dependence on the predictor display as a 
function of his proficiency in accomplishing normal 
control with conventional instruments. 

In programmed learning terms, the adaptive predictor would be a "prompt" 

which could be "faded" to bring the desired behavior under control of the 

designated stimuli generated by the conventional instruments. 

I -9- 

i . 20 



To illustrate the rationale underlying the first reason noted 
above, i. e. , that the predictor trace facilitates learning of the aircraft's 
dynamics, consider Figure 2. In Figure 2 (a) the predictor trace shows 
that, at time zero (or Tq), the pilot was climbing slightly prior to inputting 
a brief, forward control movement (dive). The predictor trace shows the . 
pilot how the aircraft will respond to his input over a continuous period of 
time. The pilot can clearly see the aircraft's response lag and can corre- 
late the kind and degree of control input with the aircraft's behavior. 

The same information is available with joint use of pitch and 
altitude indicators. However, these indicators are neither integrated, 
as is the case with the predictor trace, nor predictive of future stutes. 
At any given time, the pilot c annot see the aircraft's response lag or its 
future trajectory and he cannot easily correlate the kind and degree of 
control input with the aircraft's behavior. Obviously pilots do learn how 
their aircraft respond to control inputs. However, such learning occurs 
only after extensive, "brute force" training. We are suggesting, there- 
fore, that learning the dynamics of any aircraft may be accelerated by 
use of a predictor display during training. 

2. Exploratory Research 

This section summarizes experimental research conducted during 
Phase I. A more detailed account is presented in Appendix A. 

The research focused on determining transfer of training from 
a CRT side-looking display, having a predictor trace, to a set of conven- 
tional instruments. The instruments included altitude, pitch, glideslope, 
airspeed, percent power tachometer, and vertical velocity indicators. 
Control was accomplished with stick and power controls. The task was 
final approach to landing, as exemplified in the display shown in Figure 1. 
aircraft simulated was a Cessna T-37. 

Two groups of Subjects were used in the study. A Control Group 
of six Subjects learned to perform the final approach task without benefit 
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bEST COPT AVMLABLE 




(a) CRT with Predictor trace 



+10- 




+5. 




0- 


• 


-5- 




-10- 





+ 10- 




+5- 




0- 




-5- 




-10- 




^1 



+10- 




+5- 




0- 




-5- 




-10- 




^2 



+10- 




+5- 




0- 




-5- 




-10- 




^3 



(b) Pitch indicator at four 
points in time 



Figure 2. Modes of presenting pitch informa- 
tion and their effects on learning 
aircraft response to control move- 
ments. 
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of the predictor instrument, i. e. , using only the conventional instruments. 
The predictor display was available, in an adaptive mode, to a Predictor 
Group of six Subjects during training trials but not during test trials in 
which both groups were tested on the conventional instruments. The 
adaptive feature involved the predictor appearing for five seconds, repe- 
tively, while the aircraft was out or about to go out of the tolerance 
envelope (five seconds into the future), as shown in Figure 1. 

Performance measures were (1) percent of time within the +0. 5** 
glideslope envelope, and (2) percent of time within +6 knots airspeed 
tolerance about the 98 knot nominal airspeed. The criterion established 
for successful training performance on the task was three successive 
landings of 95 percent "within tolerance time" in terms of both glideslope 
and airspeed control. 

Unfortunately, results of this exploratory research were apparently 
highly confounded. Differences in variability between the two groups were 
extremely large, criterion score distributions for both groups were bimodal, 
and it was found, after the study was completed, that the analog computer, 
which simulated the aircraft, had become intolerably unstable. Very 
importantly also, was the finding that the 5-second on/lO-second off adaptive 
logic of the predictor instrument resulted in infrequent appearances of the 
predictor trace. In fact, its appearance was almost negligible, rendering 
the two experimental conditions more similar than different. 

Despite the gross problems, most of the Predictor Subjects out- 
performed Control Subjects in terms of criterion scores and their varia- 
bility was substantially lower. In addition, a great deal was learned that 
was applied to the full-scale experiment deacrin -jd in Sections II, III 
and rv. 
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D. RATIONALE FOR THE PHASE U EXPERIMENTAL DESIGN ' 

In addition to specific problems encountered in our exploratory 
research, the study revealed that a full-scale experiment on the effects 
of the adaptive predictor instrument on transfer of training to conven- 
tional flight instruments was somewhat premature. In fact, such an 
experiment would be out of sequence with a systematic investigation of 
the predictor display's potential contribution toward facilitating learning. 
For example, the optimum adaptive logic for the final approach task, 
as well as for other related tasks, has yet to be determined empirically. 
The ON-OFF program selected for the exploratory study appears now to 
be inappropriate by virtue of its failure to provide a predictor trace of 
useful duration or frequency. 

Similarly, the optimum prediction span for combination with an 
optimum adaptive logic also has not been determined emnirically. 

Since no adequate rationale existed for selecting appropriate adaptive 
logic or prediction span, it was believed that further arbitrary selection 
would be neither fruitful nor cost-effective. Thus, one major implication 
that emerged from^ur exploratory research was the requirement to 
optimize the predictor display parameters for our specific task. In 
particular, variations in adaptive logic and prediction spans needed to 
be evaluated. 

Considerable time was allocated to mechanizing a fruitful simulation 
of T-37 aircraft dynamics and instrumenting simulated conventional con- 
trols and displays. Too little time and analog computer capacity were 
left to manipulate critical experimenta.1 variables effectively in a nine- 
month study. 

All things considered, it was decided to run a 3 x 3 factorial experi- 
ment, having three levels of prediction span and three levels of adaptive 
logic. In addition, a tenth condition (control) was conceived for purposes 
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of establishing baseline performance by which the nine experimental 
predictor conditions could be compared. All conditions were to include 
the side-looking CRT display (Figure 1) and exclude the conventional 
instruments. Given that one or more of the predictor instrument condi- 
tions substantially improved performance on transfer over Hhe control 
condition, such a condition(s) would then be recommended as that con- 
dition to be used in a follow-on experiment employing conventional 
instruments. 

Equipment was therefore modified to meet the requirements of the 
experiment described. 
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U. EXPERIMENTAL METHOD 

A. EXPERIMENTAL DESIGN 

The experimental design was a 3 x 3 factorial with a tenth (control) 
condition. Independent groups of five Subjects were randomly assigned 
to each of the 10 conditions. 

Figure 3 shows examples of the display configurations for each of the 
nine experimental predictor conditions. The three rows of displays illus- 
trate levels of predictipn span, beginning with 5 second spans at the top, 
10 second spans in the middle and ending with 20 second spans at the 
bottom. 

The three columns reflect levels of adaptive logic. The left column 
illustrates that the predictor appears only when the aircraft symbol 
touches the tolerance envelope. The middle colunm shows that the trace 
appears when the aircraft symbol is predicted to touch the tolerance 
envelope 5 seconds into the future. Finally, the third column demonstrates 
that trace appearance occurs when the aircraft is predicted to touch the 
envelope 10 seconds into the future. 

In all cases the predictor trace remained ON as. long as the conditions 
specified above held true. 

B. EQUIPMENT 

Primary equipment for the study included a pilot's display/ control 
station, an analog computer and a computer -calculator. The display/ 
control station is shown in Figure 4. The main display was a side-looking, 
16-inch diagonal CRT, mounted so that the center of the CRT was approxi- 
mately at eye level. A glideslope of 4. 3° and a +0. 5° envelope about it were 
drawn on the face of the CRT. The display was scaled to show a 6, 666 foot 
or roughly 1-1/4 mile ground range, with an initial altitude of 500 feet at 
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BEST COPY AVAILABLE 

5 Second Prediction Spana 






20 Second Prediction Spane 




Appears when a/c is out Appears when a/c will 




of tolerance. 



ADAPTIVE MODE 0 



be out of tolerance 
5 seconds ahead. 

ADAPTIVE MODE 5 



Appears when a/c will 
be out of tolerance 
10 seconds ahead. 

ADAPTIVE MODE 10 



Figure 3. Predictor configurations for each of the nine 
experimental conditions. 
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Figure 4« Operator's display/ control station* 
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the upper right corner. Present aircraft position was represented on the 
CRT by an annular symbol. 

Directly below the main CRT display were two indicators. A percent 
engine power indicator was located on the left and an airspeed indicator 
was positioned on the right. No other displays were employed. 

Controls for the simulation consisted of a pedestal mounted joystick 
and a throttle mounted at elbow height. The joystick, adapted from an 
autopilot controller, was spring loaded, moderately self-centering, and 
traveled through an angular excursion of +20o. The oil-damped throttle 
traveled through a total arc of 60O. Full-back position represented engine 
idle (50 percent RPM), while full-forward position constituted maximum 
power (100 percent). Normal trim power setting of 82 percent was achieved 
by a throttle position 8 to 10 degrees forward of vertical. 

Generation of the predictor trace of future aircraft trajectory was 
accomplished by a fast-time, repetitively cycled analog model, based on 
a real-time T-37 aircraft model.^ This model operated with a time scale 
1000 times faster than the real-time model and was reset and operated 
15 times per second. During each reset cycle, each integrator in the 
fast-time model sampled the output value of the corresponding integrator 
in the real-time model. The fast-time model operated under the assump- 
tion that the operator would hold whatever stick position he held at the start 
of anv single prediction cycle for one second. Such an assumption is neces- 
sary for the generation of a usable prediction, but the selected value was 
somewhat arbitrarily, rather than empirically, based. 



^Equations used for the computer simulation may be found in Appendix B. 
2 

W'>rk currently underway in this Laboratory is systematically determining 
the effects on performance of different stick assumption values. 
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The model also operated under an identical assumption with respect 
to throttle' control. If either of the controls were moved during a pre- 
diction cycle, the next cycle (occurring in 0. 067 seconds) would display 
the effect of the control input changes. For all practical purposes, 
prediction changes were continuous in nature. 

As indicated, the real and fast-time models of the T-37 aircraft were 
simulated by a special purpose G. A. Philbriclranalog computer, the 
second major equipment used in the study. The third piece of equipment 
was an Olivetti Programma PI 02 computer /calculator whose function was 
to program the analog computer and print-out prescribed operator per- 
formance data on tape. 

C. SUBJECTS 

Subjects used in this study were Air Force and Navy ROTC students 
from the Universities of Loyola and California, Los Angeles. They were 
paid $10 for participating in the study. Their ages ranged from 18 to 24. 
All had little or no flying experience but met Air Force and Navy qualifica- 
tions for flight training. The Subjects were distributed randomly across 
experimental conditions with the single constraint Hiat each condition con- 
tained the same balance of flying experience. 

D. PROCEDURES 

Subjects participated in two experimental sessions of approximately 
one-hour duration each. The sessions occurred either on successive days 
or with no more than one day of separation. 

Prior to performing the task. Subjects were given instructions which 
described the task and display- control equipment in detail (Appendix C). 
Any questions were answered. Two practice runs on the simulator, one 
with and without the predictor instrument, were given. If the task was not 
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fully understood at that point, additional clarification was made until the 
Subject was completely satisfied. 

Each test session consisted of sets of six runs on the simulator, 
followed by one-minute rest periods. At the end of the first half of a 
test session, Subjects were permitted a five-minute break. 

With the exception of the control group, Subjects were given the follow- 
ing sequence of runs within a set of six: ^ 

Run 1: Training (with predictor instrument) 

Run 2: Test (without predictor instrument) 

Run 3: Training (with predictor instrument) 

Run 4: Test (without predictor instrument) 

Run 5: Training (with predictor instrument) 

Run 6: Test (without predictor instrument) 

Thus, odd and even runs were training and test trials, respectively, through- 
out a test session. 

The task given to Subjects was essentially the same as that performed 
by a pilot, where the aircraft is flown by an autopilot in all axes except 
the longitudinal. In this situation the pilot's functions are to maintain air- 
speed within limits with the power control (loading talk) and to track the 
glideslope with the attitude control (primary task). 

The computer simulation for each trial was initiated with the aircraft 
symbol in level flight. After approximately eight seconds, the symbol 
reached a point where descent was required and, subsequently, a power 
change was necessary. The glideslope /airspeed tracking task continued 
until the aircraft symbol was at an altitude of approximately one foot (a 
simulator hardware limitation). 

The simulator included an audio stall warning device which activated 
when airspeed decreased to within five knots of stall speed. The actua- 
tion of the device warned the Subject (and experimenter) that task sharing 

was not occurring to the desired extent. 
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It should be noted that performance measurements continued whether 
the aircraft stalled or not. On some occasions, in fact, the aircraft did 
stall and probably would have been lost were the situation real but con- 
tinuance of the run maintained consistency in the performance measures. 

The tracking task continued until the aircraft "touched down" on the 
"runway, " at which time the trial was terminated. Subsequently, the 
symbol disappeared and the equipment was reset for the next trial. Time 
for one trial was approximately 60 seconds. 

At the beginning and end of test sessions plots of simulator flight tracks 
were made with an x-y recorder using pre- set throttle and control inputs. 
Such calibration runs were all observed to be without non- linear anomalies. 

E. PERFORMANCE MEASURES AND STATISTICAL ANALYSES 

1 

The primary measure of performance was integrated altitude error, 
i. e. , altitude deviations about the ideal, 4. 3° glideslope. A secondary 
measure was integrated airspeed error. 

Two independent analyses of the data were performed. Of main interest 
was the analysis of the test trials which provided an indication of transfer 
effects. The second analysis was concerned with the training trials which 
permitted evaluation of the effects on performance of different design 
configurations of the predictor instrument. 

Transfer effects were determined through use of the data from the 
second experimental session only, since nearly all conditions were at or 
near asymptotic levels at the start of that session. Had asymptotes not been 
reached until the last half of the second session, for example, analysis 
would then have been performed on the data from the last half session. 

For both test and training trials three analyses of variance were con- 
ducted. Two evaluated the independent effects of prediction span and 
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adaptive mode variables. The third analysis included all ten conditions. 
Duncan's range test was selected to determine the significance of differences 
between all conditions in both analyses of variance tests. 
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A. TEST TRIALS 

1. General Learning Performance 

Figure 5 shows altitude error learning curves of the 10 experi- 
mental conditions on test trials. Each point on a given curve represents 
the mean of 12 trials X five Subjects or 60 data points. With the exception 
of Condition 1, all curves appear to have asymptoted on the last 24 trials. 

A striking feature of these learning curves is the fact that error 
scores for the Control condition and Conditions 2 and 4 were approximately 
50 percent higher than those of Conditions 3, 8 anu 9 for the last 24 trials 

and that such differences appear to have stabilized. The implication 
here is that use of certain predictor instrument configurations during 
training can effect a more rapid and, very importantly, permanent 
increase in performance on a simulated approach to landing task than 
can be accomplished without it. Such an implication is probably only 
partially correct, as will be discussed in Section IV of this report. 

In general, with the exception of Condition 2, all experimental 
conditions appeared to be superior to the Control condition. Moreover, 
most of the experimental conditions showed sharp declines in error 
scores during the first 24 trials. Condition 9 demonstrated by far the 
highest level of performance at the start* 

2. Transfer Effects 

Two one-way analyses of variance were performed on the test 
trial data (altitude and airspeed error) to derive error terms for use in 
Duncan's multiple comparison test (Edwards, I960). The Duncan test was 
subsequently used to determine the significance of differences between all 
combinations of the ten conditions. 
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12 24 36 48 

Groups of 12 Trials 

Figure 5. Mean altitude error as a function of experimental 
condition: test trials. Symbols within the par en- 
theses refer to the level of prediction span (P) 
and adaptive mode (A). 
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a. Altitude Error 

Figure 6 shows the mean integrated altitude error (in foot- 
seconds) for each of the 10 conditions on the last 24 trials. A summary 
of the ANOVA for altitude error is presented in Table 1. As can be seen, 
the main effects (conditions) was significant beyond the . 01 level. 

Results of the Duncan's range test are shown in Table 2, 
All experimental conditions except Conditions 1 , 2 and 4 were significantly 
superior to the Control condition. Moreover, Conditions 3, 6, 7, 8 and 
9 were significantly better than Conditions 1, 2, 4 and 10, with Conditions 
8 and 9 also superior to Condition 5. 

In considering the independent effects on performance of 
prediction span and adaptive mode. Figure 7 shows each variable when 
averaged over the other variable. For the variable of prediction span, 
an ANOVA yielded an F ratio of 20. 71, significant beyond the . 01 level. 
A Duncan's range test revealed that performance on each of the three 
prediction spans was significantly different (p < . 05) from the others. 
Thus, performance improved significantly on test (transfer) trials as 
prediction span increased from 5 seconds to 10 seconds to 20 seconds on 
training trials. 

With regard to adaptive mode (Figure 7b), an ANOVA yielded 
an F ratio of 11.56 which was again significant beyond the . 01 level. A 
Duncan's range test indicated that the 10 second adaptive mode was signi- 
ficantly better than the 0 and 5 second modes (p <. 01), but that modes 0 
and 5 seconds were not significantly different from each other, as is 
clearly implied in the figure. These results indicate, therefore, that the 
sooner the predictor instrument appeared on the operator's display during 
training trials -- predicting an out- of- tolerance condition -- the better 
the performance on test trials. 
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Figure 6. Mean integrated altitude error as a function of 
experimental condition (test trials). 
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Analysis of Variance of Altitude Error Scores 



Source 


df 


SS 


MS 


F 


P 


Between Conditions 
Within 


9 

1190 


2,401,038.69 
26,535,850. 60 


266, 782. 07 
22,299. 03 


11.96 


. 01 


Total 


1199 


28,936, 889.29 









An analysis of variability revealed that standard deviations 
of the means of all conditions were almost identically proportional to the 
means (Figure 8). Figure 8 is essentially the same as Figure 6 except 
that the standard deviations are much sxnaller than the means. In fact, 
they are surprisingly small for the relatively complex task employed in 
this study (note the percent size figures within the bar graph). For the 
most outstanding conditions, 3, 8 and 9, standard deviations were only 
slightly greater than one-third the size of their respective means. 

As prediction span increased from 5 to 10 to 20 seconds, 
standard deviations decreased from 149. 72 to 146. 60 to 111. 18, respec- 
tively. Similarly, standard deviations were Bznaller, the longer the 
adaptive mode, i. e. , 162. 58, 151. 93 and 92. 99 for modes 0, 5 and 
10 seconds, respectively. 

b. Airspeed Error 

Figure 9 presents the mean integrated airspeed error (in 
Knot seconds) for each of the 10 conditions on the last 24 trials. A 
summary of the ANOVA for airspeed error is given in Table 3. Signi- 
ficance of the main effects was again beyond the . 01 level. 
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Table 2 



Results of Duncan's Range Test 
(Altitude Error) 



BEST COPY AVAILABLE 



Condition 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


P5-A0 




* 


















2 


P5-A5 






















3 


P5-A10 


* 


* 




* 












* 


4 


PIO-AO 




* 


















5 


P10-A5 




* 
















* 


6 


PIO-AIO 




* 




* 












* 


7 


P20-A0 


* 


* 




* 














8 


P20-A5 


* 


* 




* 


* 










* 


9 


P20-A10 


* 


* 




* 


* 












10 


Control 























NOTE: Conditions in rows at left demonstrated a significantly lower 
altitude error (p < . 05) than conditions in columns at top if 
an asterisk appears at the junction >f row and column. For 
example, Condition 3 was significantly superior to Conditions 
1, 2, 4 and 10. 
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Figure 8. Standard deviations of the mean altitude error scores 
of Figure 6. The percent figures above the conditions 
indicate the size of the standard deviations relative to 
the associated means. 
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Figure 9. Mean integrated airspeed error as a function of 
experimental condition (test trials). 
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Table 3 

Analysis of Variance of Airspeed Error Scores 



Source 


df < 


SS 


MS 


F 


P 


Between Conditions 
Within 


9 

1190 


65,661,626.83 
469, 729.264.21 


7,295,736.31 
394,730.47 


18.48 


.01 


Total 


1199 


535,3^0,891.04 









Results of the Duncan's range test are given in Table 4. Con* 
ditions 3, 8 and 9 demonstrated the lowest airspeed errors, being signifi- 
cantly lower than all other conditions, but not significantly different from 
each other. Conditions 1, 2, 4 and 10 (control) generated the highest air- 
speed errors, with the control condition being significantly superior to 
Conditions 1 and 2. 

Figure 10 shows the effects of prediction span and adaptive 
mode independently. For the variable of prediction span, an ANOVA yielded 
an F ratio of 11. 61, which was significant beyond the . 01 level. A Duncan's 
range test indicated that performance on each of the three prediction spans 
was significantly different (p < . 05) from the others. 

Statistical tests showed essentially tlie same results for 
adaptive mode as they did for prediction span. The ANOVA yielded an F 
ratio of 13. 47 (p < . 01), while a Duncan's test showed that each of Ihe 
adaptive modes was significantly different (p < . 05) from the others. 

Variability of the distributions of airspeed scores is shown in 
Figure 11. As with altitude error scores, the correspondence between 
Figures 9 and 11, in terms of the relative differences between conditions, 
is quite high. Again, standard deviations were smaller than their respec- 
tive means, although not as small proportionally as was the case for 
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Results of Duncan's Range Test 
(Airspeed Error) 



Condition 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


P5-A0 






















2 


P5-A5 






















3 


P5-A1 0 


* 


* 




* 


« 


* 


* 
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PIO-AO 


* 


* 
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P10-A5 


> 


* 










* 
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PIO-AIO 




* 










* 
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P20-A0 


* 
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P20-A5 


* 


* 




* 




* 


* 






* 


9 


P20-A10 




* 




* 


* 


* 


* 






* 


10 


Control 


* 


* 



















NOTE: Conditions in rows at left demonstrated a significantly smaller 
airspeed error (p < . 05) than conditions in columns at top if 
an asterisk appears at the junction of row and column. 
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Figure 11. Standard deviations of the mean airspeed error scores 
of Figure 9. The percent figures above the conditions 
indicate the size of &e standard deviations relative to 
the associated means. 
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altitude error scores. Also, the standard deviations decreased in size 
as prediction apan and adaptive mode increased, i. e. , 665. 65, 654. 37 
and 517. 57 for spans 5, 10 and 20 seconds, respectively, and 658. 88, 
656. 69 and 522. 02 for adaptive modes 0, 5 and 10 seconds, respectively. 

Thus, the results of airspeed error are similar to those of 
altitude error, i. e. , as prediction span increased and as the predictor 
instrument appeared sooner during training trials, the better the per- 
formance on teat trials. 

c. Comparing Altitude and Airspeed Error Results 

Conditions 1 through 10 were ranked from best to worst on 
both altitude and airspeed error and a rank- correlation coefficient (Dixon 
& Massey, 1957) was computed to determine consistency of the two per- 
formance measures. A coefficient of . 794 was obtained, indicating a 
relatively high degree of correspondence between altitude and airspeed 
error. Contributing most (47 percent) to the failure of achieving a per- 
fect correlation was Condition 7 which was ranked fourth on altitude 
error and eighth on airspeed error. 

If we compute the average rank for aach condition (which 
assumes altitude and airspeed error to be of equal importance), the best 
to worst condition is as follows: 

Conditions 8-9, 3-6-5-7-4, 10-1-2 

where two numbers separated by a comma indicate rank ties. However, 
this analysis is presented for descriptive purposes only and should not 
be construed as a valid measure of overall performance. Rank order 
merely indicates relative order and not the absolute error generated on 
each measure. Moreover, altitude error is probably much more impor- 
tant than airspeed error for most landing circumstances. 
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B, TRAINING TRIALS 

Analysis of performance on training trials was limited to the last 
24 trials. Descriptive, rather than statistical analyses were performed 
because the latter would not appear to have particular meaning within 
the context of this study. 

1. Altitude Error 

Figure 12 shows the mean integrated altitude error scorea for 
each of the 10 conditions on training trials. Test trial means are shown 
as dotted lines for comparison. With the exceptions of Conditions 4 and 
5, training performance on the conditions was much the same as observed 
for test trials. Such an outcome was more or less expected in view of the 
fact that asymptotes had been reached by the end of the 24th trial and the 
fact that the predictor display was, during the last 24 trials, off more 
than on. 

With regard to the exceptional Conditions, 4 and 5, no readily 
available explanation would appear to account for the finding that one 
of these conditions generated superior scores during training while the 
other revealed better performance during test trials. Similar reversals 
can be seen in Figure 12 for the other conditions, but the magnitude of 
the differences suggests no more than common random error. In general, 
then, performance scores on the training trails for most of the conditions 
were, as expected, similar to those observed for test trials. 

Figure 13 shows mean performance scores for each of the 
variables, averaged over the other. Again, little difference can be 
observed between training and test trials for the three levels of prediction 
span. Larger discrepancies occurred for the 0 and 5 second adaptive 
modes. However, these differences are directly due to Conditions 4 and 
5, noted above, and easily seen in Figure 12. 
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Figure 12. Mean integrated altitude error as a function of experi- 
mental condition (training trials). 
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A further examination of the data related to individual differences 
on training and test trials. While mean altitude scores were, for all 
practical purposes, similar for both training and test trials, standard 
deviations were not. Figure 14 shows that standard deviations for 
Conditions 1,2,4 and 10 (recall that these conditions demonstrated the 
poorest performance, e. g. , see Figure 6) were extremely large, relative 
to those for the test trials. On the other hand, differences between train- 
ing and test trials for the most outstanding conditions (3, 8 and 9) were 
relatively negligible. 

A cursory examination of Figure 12 suggests that the magnitude 
of standard deviations was strongly related to adaptive mode. Computing 
the standard deviations for each mode, averaged over all prediction spans, 
yielded mean standard deviations of 335. 8, 180. 9 and 90. 8 for adaptive 
modes 0, 5 and 10 seconds, respectively. Thus, the deviation essentially 
doubled in size from 10 to 5 and again from 5 to 0 seconds. 

Differences between prediction spans were, by contrast, much 
smaller. Mean standard deviations for the 5, 10 and 20 second spans 
were 270. 1, 228.9, and 108.7, respectively. 

Analysis of standard deviations thus suggests that the earlier the 
appearance of the predictor and the longer the prediction span, the smaller 
the individual differences. 

2. Airspeed Error 

Figure 15 shows the mean integrated airspeed error scores for 
each of the 10 conditions on the last 24 training trials. Test trial means 
are also shown as dotted lines for comparison. Differences between 
training and test trials were even smaller than was the case for altitude 
error scores. For practical purposes, in fact, differences were negli- 
gible and due to random sampling fluctuations. 
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Figure 15. Mean integrated airspeed error as a function of 
experimental condition (training trials). 
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Figure 16 presents mean error scores for each of the variables, 
averaged over the other. As can be seen, curves for training and test 
trials were essentially identical. 

Examination of individual differences revealed that Conditions 
2, 5, 6 and 7 esehibited somewhat larger standard deviations on test trials 
than on training trials (Figure 17). AH other conditions showed similar 
standard deviations. In general, the tendency for test trials to have 
larger atandur^ deviaiions than training trials is the £^erorse 4af tiiat 
observed for altitude error. However, such tendencies were not repre- 
sentative of the majority of conditions, nor were they systematically 
associated with the same conditions. 
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Standard deviations of the mean airspeed error 
scores of Figure 13. 
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IV. DISCUSSION AND RECOMMENDATIONS 

BEST COPY AVAILABLE 

A. GENERAL 

The primary objective of the present study was to explore the predictor 
instrument's utility as a training aid in a relatively complex, manual con- 
trol task. Various predictor display configurations were systematically 
evaluated fox- purposes of determining optimum designs for a specific 
training application. The results of the study were quite promising, 
although more research is needed, especially in view of the fact that the 
present study appears to be the first of its kind. 

While previous experimentation has demonstrated unequivocally the 
utility of the predictor instrument as a manual control aid, the instru- 
ment has not been systematically evaluated with respect to training 
applications or specific design variables in training applications. Thus, 
this study initiated the first steps toward such an evaluation by employing 
nine configurations, representing three levels each of two variables, 
prediction span and adaptive mode. 

Conservative attitudes usually prevail in drawing conclusions from 
experimental results, often to protect against the possibility of contra- 

-dietoryHPtttu g e r ese a r ch . — Howev e r, ceiwentte nal e on se rva t ivi s m is 

sometimes set aside in light of somewhat dramatic and far-reaching 
findings. At the forefront of our results was the finding that certain 
predictor d< ^iplay configurations facilitivted learning to such an extent 
that mean performance on transfer (test) trials was considerably higher 
than that of a control condition in which the predictor dl&play was never 
used. And, it can be seen in Figure 5 that the performance curves of 
the control and several predictor display conditions appear to have 
reached asymptotes, implying that the effect of the predictor on transfer 
is a permanent one. 
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Of course, the apparent stability of the differences shown in Figure 5 
may not be permanent and, in fact, would not be expected to be permanent, 
given increased training. Certainly, operators would continue to improve 
over prolonged periods, as is the case with pilots, for example. But 
such an expectation and likelihood is clearly consistent with our hypo- 
thesis that the predictor instrument would accelerate learning. Thus, 
whether the asymptotes in Figure 5 are real or apparent, the conclusion 
is the same, namely, that the predictor display accelerated training sub- 
stantially. Only the absolute magnitude of the facilitation gain remains 
undetermined, but it appears that perhaps two to three times the number 
of trials employed in this study might be necessary before all conditions 
would approach one another. ^ 

The use of ground training simulators for pilots has become wide- 
spread because of (he almost prohibitive costs involved in extensive air- 
borne training. This fact notwithstanding, the costs of purchasing, 
operating and maintaining costly, complex ground simulators are by no 
means insignificant. On the contrary, they are substantial. The results 
of the present study suggest that training time, along with operational 
and maintenance costs, might be reduced by as much as 25 to 50 percent 
through use of an optimally configured predictor instrument. Moreover, 
{His suggestion is nof limited to pilot^ircraft systems, for a varl6ty~df 
systems contain similar complex and interrelated manual tasks that require 
lengthy periods of human operator training. 

B. SPECIFIC FINDINGS 

1. Experimental Predictor Displays 

As implied in the paragraphs above, not all predictor displays led 
to higher performance than the control condition. In general, conditions 
having the shortest prediction spans and adaptive modes exhibited somewhat 
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poorer performance than the control condition, while the opposite configu- 
rations showed superior performance. This finding was not expected, nor 
does it lend itself to easy explanation, for it was believed that even a small 
amount of prediction information would have a positive effect on performance. 

Perhaps a meaningful explanation for the differential results at 
this time is related to the concept of "facilitation-interference. " Almost 
any secondary task that is imposed on an operator simultaneously with a 
primary task may have some degree of interference with the latter. In the 
present case, periodically observing the predictor trace at the expense 
of continuously monitoring the aircraft symbol clearly might have been a 
conflicting and somewhat incompatible secondary task during training trials. 
In view of the fact that the shortest prediction spans and adaptive modes 
provided very little prediction information, it is quite possible that what- 
ever utility they may have provided an operator may have been completely 
transcended by their interference with the prinnary task. Hence, learning 
was slower than the control condition which was not confronted with the 
predictor's i Terence. 

By c j,r reasoning, it can be held that the longer prediction 
spans and adaptive modes provided operators with so much valuable infor- 

mation that their interference with the prima ry task was insi gnificant by 

contrast. 

2. Prediction Span 

Prediction spans of 5, 10 and 20 seconds were used in this study. 
For both altitude and airspeed error measures, when averaged over 
adaptive modes, perfiarTriance was significantly better the longer the spans 
used during training trials. In the present investigation, therefore, the 
further into the future the display predicts the projected position of the air- 
craft, the more information an operator has available to help him determine 
and understand the specific response characteristics of his vehicle. 
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Slopes of the performance curves (Figures 7 and 10) indicate 
that the optimum prediction span for the task and display used in this study 
was not achieved, i. e. , neither asymptotes nor U-shaped functions were 
evident. Thus, it must be presumed that spans longer than 20 seconds 
would have revealed even higher performance than that observed which, 
together with the general findings discussed earlier and described in 
Figure 5, has led us to conclude that the predictor instrument has some- 
thing more than a simple "facilitating" effect as a training aid. 

These results obviously elicit speculations as to what the optimum 
prediction span might be. Such a span should be closely related to the 
temporal aspects of the display used. For example, the display used in 
the present study required approximately 60 seconds of "flying" time 
from beginning to end of task. Thus, a 20 second span consumed one- 
third of the display flight path and progressively less of the predictor trace 
was visible after two-thirds of the display was traversed. In considering 
a longer span, say 30 seconds, visibility reduction would commence after 
only 50 percent of the display was traversed. We suspect, therefore, 
that the optimum configuration for our display lies between 20 and 30 
seconds but, in view of the lack of related research and theory in this 
area, we would not be surprised to find the optimum span to exceed our 
i5xpB?rtfftl?m^ 

3. Adaptive Mode 

Much the same discussion can be advanced for adaptive mode as 
was given for prediction span. For both altitude and airspeed error mea- 
sures, performance increased with longer adaptive modes during training 
trials (0, 5 and 10 seconds). This conclusion is only slightly tainted by 
the fact that the slope of the performance curve for altitude error between 
0 and 5 second modes was quite insignificant (Figure 7). However, because 
performance increased substantially on the 10 second mode, and because 
the airspeed error curve was consistently sloped downward (Figure 10), we 
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suspect the negligible difference found bets en 0 and 5 second modes to 
be apparent, rather than real. Examination of Figure 6 reveals that 
Condition 2 (P5-A5) was primarily responsible for the failure of the 
5 second adaptive mode to be significantly better than the 0 second mode, 
when averaged over prediction spans. That condition exhibited an unusually 
high altitude error rate. 

Figure 5 may provide some insight into the performance of Con- 
dition 2. It can be noted that its performance on the second 12 trials 
was superior to both the Control and three experimental conditions. Sub- 
sequently, Condition 2's performance decreased rather substantially 
(14 percent) on the third 12 trials and then improved again on the last 
12 trials. Unless one accepts the notion of "unlearning" as a result of 
training, this performance decrease must be considered to be due to 
extraneous factors, such as failinr^ motivation or conscientiousness on the 
part of one or more Subjects within Condition 2 and /or to equipment problems. 
In any event, the negative trend cannot be taken seriously, although it also 
cannot be discarded, unfortunately, in the overall statistical and graphic 
analyses. 

The reader will note in Figure 5 that a similar performance 
reversal was demonstrated by Condition 7 on the last 12 trials. Since this 
condition showed the highest performance on the third 12 trials, it is likely 
that it is at least comparable to Conditions 3, 8 and 9. 

4. Performance Variability 

It is usually the case that the simpler the task, whatever it might 
be, the smaller the individual differencea in performing it. All "normal" 
Subjects would be expected to perform almost equally well on an extremely 
easy iask, while a highly complex task would yield rather wide individual 
differences as a function primarily of inherent capability and task 
experience. Although the task used in this study was a simplified version 
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of that confronting a pilot during an approach to landing and must be 
considered a much easier task than the pilot's, it was, nevertheless, 
relatively complex in contrast to most tracking tasks employed in most 
laboratory studies. Since individual differences found in the latter 
research tend to be relatively broad, perhaps even wider differences 
might have been anticipated in the present study. Such was not the case, 
however. 

Standard deviations of the mean altitude and airspeed errors 
were substantially smaller than the means (Figures 8 and 11). In some 
conditions, they were only slightly larger than one-third of their respec- 
tive means. Even the control group showed relatively small standard 
deviations for altitude and airspeed errors (63 percent and 71 percent 
of the means, respectively). 

Probably the major factor contributing to relatively small standard 
deviations was the homogeneous group of Subjects employed, i. e. , ROTC 
students having physical and Intellectual qualifications for flight training. 
However, it is apparent that specific experimental conditions employing 
the pr'^dictor instrument also had much to do with depressing individual 
differences (see Conditions 3, 6, 8 and 9 in Figure 8). 

Variability is often ill-discussed in favor of mean performance. 
This is unfortunately because it may often be more important in specific 
situations. In their study on the effects of the predictor instrument on 
aircraft landing performance, for example, Wulfeck, Prosin and Burger 
(1973) found similar means for control and experimental conditions but a 
much smaller standard deviation for the predictor instrument group. The 
implication is that use of the predictor would lead to far fewer accidents 
and wave- off s than would be the case without it. In the present study, not 
only were mean performances of several predictor groups substantially 
superior to the control condition, standard deviations were also highly 
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depressed, indicating very narrow distributions about high mean perform- 
ance levels. In general, then, nearly all Subjects in these predictor 
conditions performed with accuracy and precision on transfer trials. 

5. Training Trials 

Performance on the last 24 training trials was almost exactly as 
expected, in view of the fact that performance on test trials had asymptoted, 
for the most part, at that point. Few differences could be observed between 
training and test trials for most of the conditions (Figures 12 and 15) and 
for the specific levels of prediction span and adaptive mode (Figures 13 
and 16). 

Similarities between training and test trials are easily explained, 
particularly for the high performing conditions. The more accurate an 
operator performs the task, the less frequent is the appearance of the pre- 
dictor instrument. In the extreme case, the predictor would not appear 
at all during a trial. Thus, conditions were almost identical for training 
and test trials during the last half of the experiment for several of the 
predictor instrument groups. Performance would therefore be expected 
to be identical as well. 

With regard t o airspeed error measures (Figure 15). it can be 

concluded that differences between training and test trials were negligible 
for all conditions. Some discrepancies existed, however, for altitude 
error scores (Figure 12). Here, Conditions 4 and 5 showed rather large 
differences, but in opposite directions. No explanation is readily apparent 
for either the magnitud ' of the differences or, in particular, the observed 
reversals. In both conditions, the predictor span was 2^ seconAB but it 
was ON more frequently in Condition 4, due to the large^ error accrual 
in that condition than in Condition 5 on training trials. On the other hand, 
the trace should theoretically have been ON more frequently during the 
first 24 trials because of the longer adaptive mode (5 seconds vs. 0 seconds), 
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thus leading to higher performance on the last 2A trials. However, none 
of this dialog lends much to a useful explanation. Suffice it to say that, 
in general, performance on training and test trials was very much alike, 
aa expected* 

C. IMPLICATIONS 

Heretofore, research on predictor instruments for potential use in 
manual control tasks has been limited, although the first relevant study 
was conducted well over a decade ago and yielded most promising results 
(Kelley, 1962). Literally thousands of experiments have demonstrated 
that the human operator has limited capacity to control complex vehicles 
and systems with precision. Since there are few such vehicles and 
systems which do not at least require manual control as back-up to 
automated devices, there is a clear need for devices, methods and/ or 
procedures to improve the human operator's performance. Results of 
the present study add to those of previous research which point to the 
use of a predictor instrument as a significant step towards increasing 
the manual control operator's capability to learn and control complex 
systems at a level of precision not previously believed possible « 

It OpeiNBrtt<malriBmd~TfftiMi^ of tKe^T^e^lctor Inatrumeht 



Results of this study suggest strongly that the predictor instrument 



may have considerable utility as a training aid in a wide variety of complex, 
manual control tasks. Transfer effects appear to have achieved practical, 
as well as statistical significance. Very importantly, design configura- 
tions used appear not to have been optimum for the specific task/display 
employed (see Figures 7 and 10) and, therefore, the ultimate utility of 
the predictor instrument for our task is apparently yet to be determined* 
That is, our data suggest that a more appropriately designed predictor 
display (e. g. , a predictor span of 30 seconds and an adaptive ismde of 
15 seconds) would have led to even greater transfer effects than that 
observed in this study. 
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2. Optimiaing the Predictor Instrument ggj]" QQpy 

Although our current results do not directly imply it, it seems 
quite apparent that optimum configurations of the predictor display are 
likely to be different for every major .ategory of vehicle. For example, 
the faster the vehicle, the more likely will be (he need for longer pre- 
diction spans. In the training situation, different adaptive modes may 
also prove optimum for different vehicles. Other design variables which 
were not included in this experiment would similarly require treatment 
during the optimization process. 

The present study was 11 dted to a one -dimensional tracking task, 
although the power control provided elements of a two-dimensional task 
and the display, of course, was two-dimensional. Ultimately, evaluation 
of the predictor instrument for purposes of deriving optimum designs 
should involve three-dimensional displays and models of vehicles v^ich 
account for four to six degress of freedom. 

D. RECOMMENDATIONS 

In a sense, this study represents a milestone in the investigation of 
predictor applications to manual control tasks* Evidence now is available 
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which indicates that the predictor instrument has significant utility in both 
operational and training environments. It facilitates operator performance 
and it enables operators to learn a control system more rapidly than they 
would without it« In either case, however, we have only begun to investi- 
gate the parameters which affect the design of predictor instruments and 
which undoubtedly affect the operator's performance as well. Recommen- 
dations for further research are listed below: 

• With the results of the present study in hand it would now be 
possible to evaluate transfer of training from the predictor/ 
side-looking CRT display task to one in which the Subject 
engages in an approach to landing by using only conventional 
instruments, such as the pitch indicator, glideslope indicator, 

-54- 



65 



BEST COnr AVAIUBLE 



altimeter, vertical speed indicator, airspeed indicator 
and percent power tachometer. 

• The present study showed rather wide performance 
differences between control Subjects and those in certain 
predictor configuration conditions. Such differences 
appeared to have stabilized midway through the experi- 
ment. It would be useful to conduct a longer study to 
determine the absolute extent to which the predictor 
reduces training time. Results from this study suggest 
that a training time savings of from 25 to 50 percent 
might be achievable on the particular task employed. 

• Typically, training on simulators requires close observa- 
tion by one or more instructors. This "one on one" rela- 
tion is an expensive training mode. It would be of con- 
siderable interest to determine whether a single instructor, 
equipped with several predictor displays, could monitor 
adequately the performances of several trainees simul- 
taneously. In addition tc the savings in cost that might 
accrue, a major advantage might be the diagnostic attribute 
of the predictor displays in determining the weaknesses of 
trainees, i. e. , the predictor provides forecasts of trainees' 
immediate responses which indicate correct or incorrect 
control actions. 

• Variables important to the design of predictor instruments 
need to be investigated for determininf/ optimum configura- 
tions for general classes of vehicles. Light, intermediate 

and he avy a ircr af t and se ag o i ng ve s s eitrcon^t ut e a wi de 

array of dynamics that would likely require different con- 
figurations. However, differences ar« likely to be solely 
in quantitative, rather than qualitative terms. Ideally, a 
large parametric investigation should be conducted which 
evaluates the variables across vehicle systems. Results 
from such a study would enable the development of "handbook" 
data for the design of specific predictor displays for specific 
vehicles or claeaos a{ vehicleB, 

Recent advances in computer technology, primarily developments in 
micro-miniature circuitry, indicate that computers required to generate 
complex predictor instruments can be of relatively small size and cost. 
The feasibility of installing such instruments on board small vehicles as 
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helicopters and fighter aircraft appears to be all but eetablished. It is 
of no little importance, therefo/e, that predictors be thoroughly researched 
so that ultimate utility and cost-effectiveness can be established with some 
degree of precision. 

The predictor concept has been empirically shown to be a significant 
advancement in the area of manual control. We believe that it is now time 
to investigate the limits of its worth and applications. 
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PHASE I EXPLORATORY RESEARCH 

Exploratory research was performed which examined the feasibility 
of simulating a T-37 instrument landing. In the later stages of the 
investigation, a number of Subjects participated as operators for purposes 
of evaluating the instrumentation, procedures, and one adalptive predictor 
concept. Brief descriptions of the elements of this research are presented 
in the following subsections. 

A. CONVENTIONAL INSTRUMENTATION 

The relatively large number of primary displays and controls used by 
the pilot in executiro; the final approach are listed in Table A«-l« His general 
strategy is to contioi airspeed and heading with the stick while controlling 
sink rate with the throttle. The interactions among these variables in 
actual landings result in a task that is much more complicated than one 
might expect. To keep the training and instrumentation problems within 
manageable bounds, we simplified and reduced the simulated task to that 
of longitudinal control, i. e« , glideslope tracking in a T-37 dirtied -up in 
the landing configuration. The instruments used and their locations in 
the trainer/ simulator which was modelled after the T-S? ASUPT are shown 
in Figure A-l« Note that the compass and radio, listed in Table A«-l are 
excluded. — 

Controls for the simulation consisted of a pedestal mounted joystick 

and a throttle mounted at elbow height. The joystick, adapted from an 

autopilot controller, was spring loaded, moderately self-centering, and 

traveled through an angular excursion of +20^. The oiUdamped throttle 

traveled through a total arc of 60^; full back position represented engine 

idle (50 percent rpm), while full forward position constituted maximum 

power (100 percent)^ Normal trim power setting of 82 percent was achieved 

by a throttle position 8 to 10 degrees forward of vertical. 
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Table A-1. Displays and Controls Used in IFR Final Approach 



Instruments 


Controls 


Tachometer 


Throttle 


Airspeed indicator 


Stick 


Altimeter 


Trim switch* 


Vertical velocity indicator 


Pedals* 


Attitude indicator 




Compass* 




ILS indicator 




Radio* 




*Not required for altitude and airspeed control 



(Air Training Command, 1972) 



B. ADAPTIVE PREDICTOR INSTRUMENTATION 

A fast-time, repetitively cycled analog model, based on a real-time 
T-37 aircraft model, was used to generate a prediction of future aircraft 
trajectory. This model operated with a time scale 1000 times faster than 
the real-time model and v. us reset and operated 15 times per second. 
During each reset cycle, each integrator in the fast-time model sampled th 
output value of the corresponding integrator in the real-time model. The 
fast-time model operated under the assumption that the pilot would main- 
tain whatever control stick position he held at the start of any single pre- 
diction cycle for the duration of a 10-second prediction. Such an assump- 
tion is necessary for the generation of a usable prediction, but the selected 
value of 10 seconds was somewhat arbitrarily, rather than empirically, 
based. 

The model also operated under an identical assumption relating to 
the throttle control. 
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Figure A-1. Photograph of simulated instrument 
panel and predictor display with 
t' throttle (left) and control stick (center). 
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If either of the controls were moved during a prediction cycle^ the 
next cycle (occurring in • 067 seconds) would display the effect of the 
control input changes. For all practical purposes, prediction changes 
were continuous in nature. 

An annular synxbol showing present aircraft position with aircraft 
trajectory prediction from it was presented as an outside in, side looking 
display on a l6-inch diagonal CRT mounted immediately above the con- 
ventional instrument array (see Figure 1 on Page 6)* The display 
was scaled to show a 6,666 ft or roughly Nl/4 mile ground range with 
an initial altitude of 500 feet at the upper right corner. The glideslope 
of 4, 30 and a plus and minus 0, 5o envelope about it were drawn on the 
face of the CRT. A 10*second predictor trace indicatinp predicted 
trajectory extended approximately two inches from the left side of the 
aircraft symbol and varied slightly in length as airspeed varied. 

The adaptive feature of the predictor instrument was represented 
by the appearance of the predictor display when the aircraft symbol 
departed from the ^0. 5^ envelope or when the predictor indicated that 
the aircraft would depart within five seconds and its subsequent dis- 
appearance after a preselected period. Specifically, a program of five 
seconds on and 10 seconds off was selected for the display while the 
aircraft was out or about to go out of tolerance. The display did not 
appear during periods in which the aircraft remained within the envelope. 

C. PERFORMANCE MEASURES AND CRITERION 

Two measures of performance on each attempted approach were 
recorded: percent of time within the jfO. 5^ glideslope envelope; and per- 
cent of time within +6 knots airspeed tolerance about the 98 knot nominal 
airspeed. Both measures were cumulated and printed-out by an analog* 
digital system. A secondary performance measure was number of air- 
speed failures, i. e. , any time the speed of the aircraft fell 18 knots below 
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the nominal 98 knots, the "flight" was considered a failure since the 
aircraft would stall at such an airspeed. 

The criterion for successful training performance on the glideslope 
task was three successive landings of 95 percent ^'within tolerance time'* 
in terms of both glideslope and airspeed control. 

D. EXPERIMENTAL DESIGN 

The purpose of the experiment was to determine whether or not the 
predictor instrument used in an adaptive mode would reduce the number 
of training trials required to achieve criterion on the approach task. 
Two experimental conditions constituted the exploratory experimental 
design. A Control Group learned to perform the task without benefit of 
the predictor instrument. The predictor display was available, ''aa needed^' 
to a Predictor Group during training trials but not during test trials in which 
both groups were tested on the conventional instruments. 

E. SUBJECTS 

Twelve Subjects , six in the Control Group and six in the Predictor 
Group, completed the experiment. Several Subjects dropped out because 
of frustration with the task. All Subjects were students at local colleges 
who volunteered from a pool comprised of candidates who had scored above 
the norms for college students on the Guilford-Zimmerman Aptitude Survey, 
Parts IV and V, and on the Witkin Group Embedded Figures Test. None 
had had previous flight training. 

F. PROCEDURES 

Written instructions were presented to the Subjects describing the . 

instrumentation^ their taski scoring procedures and the training goal or 
criterion. The Predictor Group^s instructions differed from the Control 
Group's only in that information describing the predictor instrument and 
its use was also included. 
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Subjects were administered an average of 48 trials per day for four 
days (sessions). Within a session, trials were sequenced such that one 
test trial immediately, followed five training trials. Short interpolated 
rest periods occurred at the end of each set of 12 trials except for those 
Subjects who "passed" a test trial. In those cases, additional test trials 
were administered until the Subjects reached criterion or failed a test. 

Each trial began with the aircraft positioned at the same range. 
However, three different initial altitudes were used: 0.49° above, 0.49° 
below and on the desired glideslope. Altitudes were sequenced such that 
all three were administered when criterion was achieved. Performance 
scores were available following each trial and Subjects were given 
immediate knowledge of their performance. 

A payment of $25. 00 was made to each Subject who completed the 
study and an additional $10. 00 bonus was offered and given to the Subject 
in each group who reached criterion within the least number of trials. 

G. RESULTS 

The results of the exploratory study were apparently confounded by a 
number of factors. First, Subjects who completed the study exhibited 
unmanageable variations in performance and the variability in criterion 
performance among them waj unusually large. For example, the standard 
deviation for Control Subjects was more than 300 percent larger than it 
was for the Predictor Group. Moreover, the criterion score distributions 
for both the Control and Predictor Groups were strongly bimodal. It was 
considered inappropriate to evaluate the data statistically, even though 
they tended to support the use of the predictor instrument as a training aid. 

A second factor that appeared to influence results of the study was 
associated with the analog computer and the simulated conventional glide- 
~~ :^ slope control instrumentation. Although relatively accurate for most 
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purposes and properly calibrated before the experiment, the computer was 
found after the experiment to have become differentially stable in driving 
the simulated conventional display instruments in relation to the predictor 
display. Such instability ma/ have varied over the course of the study and 
it can not be assumed that its effects were distributed equally across 
Control and Predictor Subjects. 

The adaptive logic employed in the predictor condition may have been 
a third factor that Influenced results. No empirical basis or developed 
rationale exists for selecting an appropriate ON-OFF program. The 
5 second ON - 10 second OFF sequencing of the predictor trace was chosen 
arbitrarily so that the predictor would only prompt the student without 
allowing him to become dependent on it. Since the OFF mode tended to 
dominate a trial with respect to time, the program probably reduced 
considerably the expected differences in conditions between Control and 
Predictor Groups. 

In summary, data from the exploratory study seemed clearly affected 
by factors unrelated to the specific conditions which were intended to 
differentiate between Predictor and Control Groups. Such factors appar- 
ently overwhelmed differences that may have occurred and could not be 
extracted by statistical process. Even so, most of the Predictor Subjects 
out-performed Control Subjects in terms of criterion scores, and airspeed 
, failures, and their variability was substantially lower. Thus, while con- 
ducting and reporting statistical analyses would be inconsistent with rigorous 
scientific procedure, the predictor instrument nevertheless demonstrated 
utility in transfer of training despite the confounding variables. 
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APPENDIX B 
COMPUTER SIMULATION EQUATIONS 



Glideslope Parameters 
Air Speed = 98 knots 

= 165.4 fps = V 



0 

.•. Vq = 2.736 x 10^ ft^/sec^ 



Altitude at 0. 75 mi. = 300 ft. 

Glideslope angles tan"^ = 4.3° 

Nominal pitch angle 9^ = 5° 

.*. nominal angle of attack = 9. 3° or = 0. 162 rad. 
= 165.4 cos 5° = 164.8 fps 
= tan = 164.8 tan 9.3° = 26.98 fps 

Sink Rate (two methods) 

1. SR = cose - sine = 12.51 fps 

or 

2. SR =— JL_ = 12. 53 fps 

^TD^^O 
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Scale Factors (ref = 50 volts) 



"max 




200 fps 




[0.25 Vxg] 


^max 




50 fps 




[VzbI 


^max 


— 


0.2 rad/sec 




[250 Q] 






0.333 rad 




[150«] 


^^Imax 




200 fps 




[0. 25 Vxj] 


nnax 




50 fps 




[VZj] 


^Imax 




6666.667 ft 




[7.5 x 10"^Xj] 


^max 




501.3 ft 




[0. 1 Zt ] 
^max"* 


*max 




0.25 rad (14.3°) 




[200a] 
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Vxi = cos 50 + Vz3<» (Vxj^ = 165.4 fps) 

VZi = cos 50 - y^^e (Vzj^ = 12.5 fps) 



- yVx^ dt 



Xt = / dt Xj = 6666. 667 ft 



^0 



.-3, 



, >.| 7. 5 X 10 10. 25 Vx«] X . 9962 
[7.5 X 10"^X ' ' ' - 



.25 



7.5x lO'VzoKlSOd] 50 

+ ^ 

150 E 



dt 



y =/|o. 



0.0024 — ?z 



[7.5 X 10""^ X ] =/ 0.0299 [0.25 Vx„] + 0.0024 



dt 



r\ 0. 1[0.25 Vx«][l50d] 50 

[0. 1 Z ] = / 0. l[Vz„] X . 9962 5 | dt 

V I 0.25 X 150 |50| 



[0.1 Zj] =yjo.09962[V2g] - 0.133 



[0.25 Vxb][1500] 



dt 
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Pitch Equation 



C = 0. 057 - 0. 027 
0 (13) (14) 



C^, = - 1.003/rad 
(14) 



Ma 



0.0171/deg 
^^U = -4^ Q <57.3) = - 15.59Q 



[250Q] = 250 X 9. 33yjo. 03 - ^^^^^ . O^Oin^ 



15.59[25Q] 
^50 



dt 



[250Q] =y|69. 98 - 11. 7 [200a] + 19.94[2«J - 154.8 [250 Q]| dt 
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B 



QVx3 = 32.2 COS 50 + ^ 



dt 



Q + 32.08 + 37. 9 Cj^ 



dt 



C = C- + a C, + « C, 



C = - (0.256 + .3856) - 4.503 a + 0.00572 6 
(17) (17) (9) • 



B 



=/l 



[250Q][0.25Vx3]50 



f^Z ^ = / 1 ^z:^^ + 32. 08 - 37. 9 X . 6416 

250 X .25 HoI 



37. 9 x4. 503 [200a] , 37. 9 x . 00572 [2^ 
200 2 



dt 



[Vzb3 



8[250Q][0.25Vx3] 

" go] 



+ 7.76 - 0.8533 [200o] 



+ 0. 108 [2 a ] 



dt 
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P = 2.35 X 10* (slugs/ft air denBit/ sea level) 

S = 183.9 ft^ 
c » 5. 58 ft. 
b = 33.80 ft 
W = 5000 lbs 

-—-=.21608 



0S 2 

~V = 5912.02 



M = 156.25 or 155.28 

lyy = lyy + lyy 

"owe "fuse 



= 3484.46 + 0.09415 x 

F 



= 3484.46 + 0.09415 x 547 = 3535. 96 

= 3536 
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SUBJECT INSTRUCTIONS 

* 



General instructions were administered to all Subjects. Subsequently, 
supplementary instructions were given to the nine experimental groups, 
describing the predictor instrument and how to use it. 

A, GENERAL INSTRUCTIONS 

The following instructions were read by all Subjects: 

This experiment is concerned with various ways flight informa- 
tion can be presented to a pilot during a particular aircraft 
maneuver -- that is, an approach to a landing. The aircraft 
is represented as a simplified computer simulation of a twin- 
engine jet trainer, the T-37. 

To operate Che aircraft, you will have four displays and two 
controls, as shown in the figure below. 






Airspeed 



Engine Power (%) 



® Audio Stall Warning 



Power 
Control 
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The primary display is the large television screen. Imprinted 
on to the screen is a black line, indicating the ideal path for 
the aircraft to follow, and two red lines, outside of which the 
aircraft is dangerously high or low. Your main task will be 
to try and keep the aircraft, represented by a small circle, 
as close to the black line as you can. 

Below the television screen is an airspeed display, having a 
scale of 0 to 120 knots. Your secondary task will be to keep 
the aircraft's speed between 90 and UO knots. If the airspeed 
.drops below 80 knots, the landing will be considered a failure 
because a real aircraft would stall out at that speed and crash. 
However, while we will automatically record stalls, you will 
continae to guide the aircraft to the bottom of the bl»ck litie. 

The simulator is equipped with an audio warning device which 
activates at airspeeds at and below 80 knots. This warning 
indicates that the airspeed is dangerously close to stall speed. 
Airspeed can be increased by applying forward pressure on the 
attitude control, increasing power, or both. 

On the left of the display panel is an engine po,\7er indicator. 
On normal approaches, when the aircraft is on speed and the 
aircraft symbol has been tracking fairly close to the desired 
flight path, you will iind that about 88 percent power ^111 main- 
tain the airspeed desired. 

The attitude control will be located between your legs when 
seated at the "cockpit" console. This control allows you to 
bring the aircraft nose upward and downward. Slight move- 
ment of this stick backwards (towards you) will cause tlie 
^ aircraft symbol on the display to climb. Similarly, slight 

forward movement of the stick (away from you) will cause the 
aircraft symbol to descend. 

f^B w-.th an automobile starting up a steep grade, the aircraft's 
speed will decrease when climbing unless some power is added. 
Increased power can be obtained by moving the power control 
forward (away from you). Similarly, when the aircraft stops 
climbing and begins to descend, some power should be removed 
or the aircraft ^ill begin to accelerate. 

I 

The task which you are about to perform can be described very 
simply. You will be asked to land the aircraft many times. 
Each landing is called a "run. " Prior to each run, the displays 
will indicate that the aircraft is in level flight at constant speed. 

-73- 

u . ■ ■ 

Er|c 84 ^ 



BEST am mum 



The attitude and power controls will already be adjusted to 
maintain level flight and constant speed, and were you not 
to touch either of the controls, the aircraft would remain in 
this stable state and fly straight ahead. When the display 
indicates that it is time for descent, you will manipulate the 
two controls such that the aircraft symbol follows the desired 
flight path (black line) and the airspeed display shows that 
your airspeed is within a safe range. 

You will be given two unmeasured runs on the simulator so 
that you can become familiar with the displays and controls. 
If you have any questions, this is the time to get clarification. 

We fuilyr realize that you have not had ^oxperiencfi oa this ty:f^ 
of task. That^s why we chose you. All that we ask is that you 
do ^e best you can. 

The overall task will consist of sets of landings, with short 
rest periods between sets. 



B. SUPPLEMENTARY INSTRUCTIONS 

Supplementary instructions given to the experimental groups were 
slightly different from each other, reflecting the different predictor 
instrument ccmfigurations. Therefore, instructions given to Condition 4 
are presented below as a typical example ai that used for all experi- 
mental groups. 

An imp!o>rtant part of this experiment is the evaluation of a 
television display feature which tells you in advance where 
ywur aircraft will be if you make no changes in your attitude 
and power controls. This prediction feature is represented 
by a liaxe projected from the aircraft symbol, as shown below: 
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The predictor line shown indicates that, with no further con- 
trol movements, the aircraft will continue to climb slightly 
and then start to descend. The end of the line is where the 
airf?rj\ft will be 10 seconds from now. To control the aircraft 
mpvft effectively, you should manipulate the attitude control 
such that the end of the predictor line touches the center black 
line. 

The predictor line will appear only when the aircraft symbol 
is touching or 's outside the red lines. When the aircraft 
returns within tiiie red lines, the predictor line will disappear. 

In the course of this experiment, you will be given one run 
with the predictor Jip0. oxm Ttm without it, one run with the 
predictor, and so on. 
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